Protection for remote sensing object detection datasets based on backdoor watermarking and region of interest encryption

doi:10.11947/j. AGCS.2024.20240092.

Abstract

Abstract:

The collection, cleansing, and annotation processes of high-quality remote sensing datasets typically entail substantial costs. Therefore, the remote sensing datasets can be regarded as intellectual properties. However, remote sensing datasets also face threats such as theft, unauthorized usage and redistribution. In order to safeguard the copyright of datasets, we propose an object detection dataset protection method based on backdoor watermarking and region of interest (ROI) encryption. The algorithm embeds object-generation watermark triggers into the original dataset and utilizes an ROI encryption algorithm to encrypt the dataset. During the watermark embedding phase, random samples are selected from the original dataset, and the triggers are embedded into random positions within the samples. During the dataset encryption phase, the ROIs in the annotation files are first initially encrypted. Then, disturbances are added within the encrypted ROIs. Finally, a unique random key is generated for each user based on a hash function, and perform secondary encryption on the initially encrypted annotation files. During the dataset decryption phase, only authorized users can decrypt the encrypted dataset, where the encrypted annotation files are restored to correct ROIs. Thereby obtaining the decrypted legitimate dataset. In the phase of asserting copyright on suspected models, a watermark test set is constructed with the object-generation watermark triggers. This test set is then inputted into the suspected model for prediction. If the watermark prediction success rate exceeds a preset threshold, it is considered that the model has utilized the protected dataset during training. Extensive experiments have demonstrated that this method effectively protects dataset copyrights without compromising dataset quality. The watermarking algorithm exhibits strong robustness against fine-tuning attacks and pruning attacks.

Key words: remote sensing dataset for object detection, dataset protection, copyright protection, ROI encryption, object-generation watermark

CLC Number:

P208

Weitong CHEN, Xin XU, Changqing ZHU, Na REN. Protection for remote sensing object detection datasets based on backdoor watermarking and region of interest encryption[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(11): 2086-2098.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

References 36

[1]	黄启灏, 靳国旺, 熊新, 等. 通道剪枝与知识蒸馏相结合的轻量化SAR目标检测[J]. 测绘学报, 2024, 53(4): 712-723. DOI:. doi: 10.11947/j.AGCS.2024.20220605
	HUANG Qihao, JIN Guowang, XIONG Xin, et al. Lightweight SAR target detection based on channel pruning and knowledge distillation[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(4): 712-723. DOI:. doi: 10.11947/j.AGCS.2024.20220605
[2]	董志鹏. 基于卷积神经网络的高分辨率遥感影像目标检测方法研究[J]. 测绘学报, 2023, 52(9): 1613. DOI:. doi: 10.11947/j.AGCS.2023.20220234
	DONG Zhipeng. Research on object detection in high resolution remote sensing imagery based on convolutional neural networks[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(9): 1613. DOI:. doi: 10.11947/j.AGCS.2023.20220234
[3]	CHEN Chen, LI Shuangjiang, XU Yongyang, et al. Correg-Yolov3: a method for dense buildings detection in high-resolution remote sensing images[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(2): 51-61.
[4]	SUN Long, WU Tao, SUN Guangcai, et al. Object detection research of SAR image using improved faster region-based convolutional neural network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(3): 18-28.
[5]	XUE M, WU Y, ZHANG Y, et al. Dataset authorization control: protect the intellectual property of dataset via reversible feature space adversarial examples[J]. Applied Intelligence, 2023, 53(6): 7298-7309.
[6]	朱长青. 地理数据数字水印和加密控制技术研究进展[J]. 测绘学报, 2017, 46(10): 1609-1619. DOI:. doi: 10.11947/j.AGCS.2017.20170301
	ZHU Changqing. Research progresses in digital watermarking and encryption control for geographical data[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10): 1609-1619. DOI:. doi: 10.11947/j.AGCS.2017.20170301
[7]	ZHU P, JIANG Z, ZHANG J, et al. Remote sensing image watermarking based on motion blur degeneration and restoration model[J]. Optik, 2021, 248: 168018.
[8]	YUAN Guanghui, HAO Qi. Digital watermarking secure scheme for remote sensing image protection[J]. China Communications, 2020, 17(4): 88-98.
[9]	LI Y, ZHU M, YANG X, et al. Black-box dataset ownership verification via backdoor watermarking[J]. IEEE Transactions on Information Forensics and Security, 2023, 18: 2318-2332.
[10]	GU T, LIU K, DOLAN-GAVITT B, et al. Badnets: evaluating backdooring attacks on deep neural networks[J]. IEEE Access, 2019, 7: 47230-47244.
[11]	LI Y, JIANG Y, LI Z, et al. Backdoor learning: a survey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022: 35(1): 5-22.
[12]	LI Y, BAI Y, JIANG Y, et al. Untargeted backdoor watermark: towards harmless and stealthy dataset copyright protection[C]//Proceedings of 2022 Conference on Neural Information Processing Systems. New Orleans: Curran Associates, 2022: 13238-13250.
[13]	HE Y, SHEN Z, CUI P. Towards non-IID image classification: a dataset and baselines[J]. Pattern Recognition, 2021, 110: 107383.
[14]	郭晶晶, 刘玖樽, 马勇, 等. 基于模型水印的联邦学习后门攻击防御方法[J]. 计算机学报, 2024, 47(3): 662-676.
	GUO Jingjing, LIU Jiuzun, MA Yong, et al. Defense method against backdoor attacks in federated learning based on model watermarking[J]. Journal of Computer Research and Development, 2024, 47(3): 662-676.
[15]	DONG L, QIU J, FU Z, et al. Stealthy dynamic backdoor attack against neural networks for image classification[J]. Applied Soft Computing, 2023, 149: 110993.
[16]	SAHA A, SUBRAMANYA A, PIRSIAVASH H. Hidden trigger backdoor attacks[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York: AAAI Press, 2020: 11957-11965.
[17]	LI Y, LI Y, WU B, et al. Invisible backdoor attack with sample-specific triggers[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 16463-16472.
[18]	CHAN S H, DONG Y, ZHU J, et al. Baddet: backdoor attacks on object detection[C]//Proceedings of the 17th European Conference on Computer Vision. Cham: Springer Nature, 2022: 396-412.
[19]	LI K, WAN G, CHENG G, et al. Object detection in optical remote sensing images: a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159: 296-307.
[20]	CHENG G, HAN J. A survey on object detection in optical remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 117: 11-28.
[21]	WANG C, BAI X, WANG S, et al. Multiscale visual attention networks for object detection in VHR remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 16(2): 310-314.
[22]	LONG Y, GONG Y, XIAO Z, et al. Accurate object localization in remote sensing images based on convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5): 2486-2498.
[23]	XIAO Z, LIU Q, TANG G, et al. Elliptic Fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images[J]. International Journal of Remote Sensing, 2015, 36(2): 618-644.
[24]	ZOU Z, SHI Z. Random access memories: a new paradigm for target detection in high resolution aerial remote sensing images[J]. IEEE Transactions on Image Processing, 2017, 27(3): 1100-1111.
[25]	WU X, SAHOO D, HOI S C H. Recent advances in deep learning for object detection[J]. Neurocomputing, 2020, 396: 39-64.
[26]	TAJBAKHSH N, SHIN J Y, GURUDU S R, et al. Convolutional neural networks for medical image analysis: full training or fine tuning?[J]. IEEE transactions on medical imaging, 2016, 35(5): 1299-1312.
[27]	CETINIC E, LIPIC T, GRGIC S. Fine-tuning convolutional neural networks for fine art classification[J]. Expert Systems with Applications, 2018, 114: 107-118.
[28]	BLALOCK D, GONZALEZ O J J, FRANKLE J, et al. What is the state of neural network pruning?[C]//Proceedings of the 3th Conference on Machine Learning and Systems. Austin: [s.n.], 2020: 129-146.
[29]	张玉, 武海, 林凡超, 等. 图像识别中的深度学习模型剪枝技术[J]. 南京理工大学学报, 2023, 47(5): 699-707.
	ZHANG Yu, WU Hai, LIN Fanchao, et al. Pruning techniques for deep learning models in image recognition[J]. Journal of Nanjing University of Science and Technology, 2023, 47(5): 699-707.
[30]	KAVIANI S, SHAMSHIRI S, SOHN I. A defense method against backdoor attacks on neural networks[J]. Expert Systems with Applications, 2023, 213: 118990.
[31]	QIU H, ZENG Y, GUO S, et al. Deepsweep: an evaluation framework for mitigating DNN backdoor attacks using data augmentation[C]//Proceedings of the 16th ACM Asia Conference on Computer and Communications Security. Hong Kong: ACM Press, 2021: 363-377.
[32]	LI Y, LYU X, KOREN N, et al. Anti-backdoor learning: training clean models on poisoned data[EB/OL] [2023-11-05]. https://proceedings.neurips.cc/paper/2021/hash/7d38b1e9bd793d3f45e0e212a729a93c-Abstract.html.
[33]	LIU Y, FAN M, CHEN C, et al. Backdoor defense with machine unlearning[C]//Proceedings of the 41th IEEE Conference on Computer Communications. London: IEEE, 2022: 280-289.
[34]	HUANG Kunzhe, LI Yiming, WU Baoyuan, et al. Backdoor defensevia decoupling the training process[EB/OL]. [2022-12-22]. https://arxiv.org/abs/2202.03423v1.
[35]	ZHANG Y F, REN W, ZHANG Z, et al. Focal and efficient IOU loss for accurate bounding box regression[J]. Neurocomputing, 2022, 506: 146-157.
[36]	YANG X, YAN J, LIAO W, et al. Scrdet++: detecting small, cluttered and rotated objects via instance-level feature denoising and rotation loss smoothing[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(2): 2384-2399.